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Declaration: Inventorship (only for 
the purposes of the designation of 
the United States of America) 
Declaration of inventorship (Rules 
4.17(iv) and 51bis.1(a)(iv)) for the 
purposes of the designation of the 
United States of America: 




I hereby declare that I believe I am the 
original, first and sole (if only one 
inventor is listed below) or joint (if 
more than one inventor is listed below) 
inventor of the subject matter which is 
claimed and for which a patent is 
sought . 

This declaration is directed to the 
international application of which it 
forms a part (if filing declaration with 
application) . 

I hereby declare that my residence, 
bailing address, and citizenship are as 
stated next to my name . 

I hereby state that I have reviewed and 
understand the contents of the 
above-identified international 
application, including the claims of 
said application. I have identified in 
the request of said application, in 
compliance with PCT Rule 4.10, any claim 
to foreign priority, and ± have 
identified below, under the heading 
"Prior Applications," by application 
number,, country or. Member of the World 
Tirade Organization, day, month' and year 
°f filing, any application for a patent 
or inventor's certificate filed in a 
country other than the United States of 
America, including any PCT international 
application designating at least one 
country other than the United States of 
America, having a filing date before 
that of the application on which foreign 
priority is claimed. 
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I hereby acknowledge the duty to 
disclose information that is known by me 
to be material to patentability as 
defined by 37 C.F.R. § 1.56, including 
for continuation-in-part applications, 
material information which became 
available between the filing date of the 
prior application and the PCT 
international filing date of the 
continuation-in-part application . 
I hereby declare that all statements 
made herein of my own knowledge are true 
and that all statements made on 
information and belief are believed to 
be true; and further that these 
statements were made with the knowledge 
that willful false statements and the 
like so made are punishable by fine or 
imprisonment, or both, under Section 
1001 of Title 18 of the United States 
Code and that such willful false 
statements may jeopardize the validity 
of the application or any patent issued 
thereon . 
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or country) 
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inventor's Signature: 
Of not contained in the request, or if 
declaration is corrected or added under 
Rule 26ter after the filing of the 
international application. The signature 
must be that of the inventor, not that of 
the agent) 
Date: 

(of signature which is not contained in 
the request or of the declaration that is 
corrected or added under Rule 26ter 
after the filing of the international 
application' 
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Malmp , Sweden 
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DEVICE FOR GENERATING SPEECH, APPARATUS CONNECTABLE TO OR 
INCORPORATING SUCH A DEVICE, AND COMPUTER PROGRAM 



PRODUCT THEREFOR 




5 Field of invention 

The present invention relates to a device for generating speech associated 
with information shown on a display, especially displays on portable devices such 
as mobile telephones and the like. A conversion circuit converts the data shown to 
audible speech helping the user to operate the apparatus. The invention also relates 
10 to an apparatus arranged to cooperate with such a device or incorporating such a 
device, and a computer program product therefor. 

State of the art 

In portable devices such as mobile telephones etc. the displays are used to 
1 5 display menus controlling the operation and settings of the device or other 

information relating to messages or games. The displays are often small, which may 
be a problem for the user, especially if he is visually impaired. Also for other 
reasons, there is a need for an audible version of the display. 

The present invention solves this problem by transforming the information 
20 displayed to audible speech. 

Summary of the invention 

In a first aspect, the invention provides, a device for generating speech, 
wherein a microcontroller is connectable to an apparatus for receiving data to be 
25 converted to speech, and sending Ihe data to a conversion circuit; and a conversion 
circuit connectable to a speaker system for converting the data to a speech signal. 

Preferably, the data is supplied as ASCII characters. 

3 0 Suitably, the conversion circuit supports various selectable languages and the 

conversion circuit is capable of downloading languages via the connected apparatus. 

Suitably, the conversion circuit supports various selectable voices and the 
conversion circuit is capable of downloading voices via the connected apparatus 

35 

Preferably, the speed of the speech signal is adjustable. 

Preferably, the microcontroller is connectable to a memory containing language 
information, such as various languages, abbreviation lists and dictionaries. 
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Preferably, the microcontroller is connectable to a memory containing voice 
settings. 

5 Suitably, the microcontroller is connectable to the apparatus by means of a system 
connector having an interface for audio signals, serial channels, power leads and 
analog and digital ground leads. 

The device may be implemented as a functional cover, comprising a shell covering 
10 the front of the apparatus and a microprocessor cooperating with the processor of 
the apparatus. 

The connectable apparatus may be a portable telephone, a pager, a communicator or 
an electronic organiser. 

15 

In a second aspect, the invention provides an apparatus having a display for 
showing various readable data, wherein a control unit is arranged to extract readable 
data for sending to a device for generating speech as mentioned above. 

20 The readable data may include texts from menus, text messages, help information, 
calendars or confirmation of actions taken with the apparatus. 

Suitably, the control unit is arranged to extract a part of the readable data, such as a 
line or a word, at a time from the display and sending it automatically to the speech 
25 generating device at a fixed or controllable rate, and/or the control unit is arranged 
to extract a line at a time from the display and sending it to the speech generating 
device in dependence of scrolling in the display. 

Suitably, the control unit is also arranged to extract a part of the readable data, such 
30 as a character, a line or a word, at a time from the display and sending it to the 
speech generating device in dependence of inputting characters to the apparatus. 

Then, the control unit may be arranged to send readable data as triggered by the 
input of definite characters, such as letters, signs, spaces or punctuation marks. 

35 

Preferably, the control unit is arranged to extract readable data from a selected file 
and sending it automatically to the speech generating device at a fixed or 
controllable rate. 
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In a third aspect, the invention provides an apparatus having a display for showing 
various readable data, including a control unit and a device for generating speech 
comprising a conversion circuit for converting data to a speech signal and 
connectable to a speaker system, wherein the control unit is arranged to extract 
5 readable data for sending to the speech generating device. 



The speaker system may be integrated with the apparatus. 




Suitably, the data is supplied as ASCII characters. 

10 

Suitably, the conversion circuit supports various selectable languages, and is 
capable of downloading languages. 

Suitably, the conversion circuit supports various selectable voices, and is capable of 
15 downloading voices. 

Preferably, the speed of the speech signal is adjustable. 

Suitably, the apparatus is connectable to a memory containing language 
20 information, such as various languages, abbreviation lists and dictionaries. 

Suitably, the apparatus is connectable to a memory containing voice settings. 

Preferably, the readable data includes texts from menus, text messages, help 
25 information, calendars or confirmation of actions taken with the apparatus. 

Suitably, the control unit is arranged to extract a part of the readable data, such as a 
line or a word, at a time from Ihe display and sending it automatically to the speech 
generating device at a fixed or controllable rate, and/or the control unit is arranged 
30 to extract a line at a time from the display and sending it to the speech generating 
device in dependence of scrolling in the display. 

Suitably, the control unit is arranged to extract a part of the readable data, such as a 
character, a line or a word, at a time from the display and sending it to the speech 
35 generating device in dependence of inputting characters to the apparatus. 

Then, the control unit may be arranged to send readable data as triggered by the 
input of definite characters, such as letters, signs, spaces or punctuation marks. 
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Preferably, the control unit is arranged to extract readable data from a selected file 
and sending it automatically to the speech generating device at a fixed or 
controllable rate. 

5 The apparatus may be a portable telephone, a pager, a communicator or an 
electronic organiser. 

In a fourth aspect, the invention provides a computer program product loadable into 
the internal memory of an apparatus having a display for showing various readable 
10 data, wherein the computer program product comprises software code portions to 
achieve the functionality of the apparatus as mentioned above. 

The computer program product may be embodied on a computer readable medium. 

15 Brief description of the drawings 

Embodiments of the invention will be described in detail below with 
reference to the accompanying drawings, of which: 

fig. 1 is a block diagram of the main blocks of the invention, 

fig. 2 is a perspective view of a system connector, 
20 fig. 3 is a data flow diagram, and 

fig. 4 is an example of a mobile phone using the present invention. 

Detailed description of preferred embodiments 

The invention will be described in relation to a mobile phone including text- 
25 to-speech conversion. The invention is also applicable in many other devices, e.g. 

pagers, communicators, electronic organisers and the like portable devices. 

Text-to-speech conversion is a feature that is of interest in many different 

areas and applications. One of the more interesting is the use in mobile phones. 

Today mobile phones are used by almost everyone and a feature like this can be an 
30 important aid, especially for the visually impaired and for users who need to focus 

on other things while using the phone, for instance car drivers using hands-free 

equipment. The text-to-speech conversion is done in hardware with a text-to-speech 

circuit. A highlighted menu label, an SMS or other readable data are sent to a 

microcontroller. The data may be received as ASCII characters and these are 
35 forwarded to the text-to-speech circuit by the microcontroller. The text-to-speech 

circuit converts the characters to audio signals and sends them to a loudspeaker 

system. 
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The invention makes the mobile telephone more user-friendly by reading 
messages and menus to help the user locate himself while browsing the menus 
system. 

Fig. 1 shows an embodiment of the invention in which the speech generating 
5 device is implemented as an accessory. The accessory is to be attached to a mobile 
phone 1 via its system connector. The accessory may be implemented as a so called 
active or functional cover, that is a shell covering e.g. the front of the phone and 
also connected to the phone's system connector. The functional cover contains a 
microprocessor holding additional functions and cooperating with the processor of 
10 the telephone. Thus, the actual outer shape of the accessory depends on the mobile 
phone and is not shown here. 

The speech generating device 5 is shown within the dashed square and 
includes a microcontroller 6 receiving the data to be converted from the mobile 
phone and passing it to a text-to-speech (TTS) circuit 7. The TTS circuit 7 converts 
15 the text to audio signals and sends them via an (optional) amplifier 8 to a 
loudspeaker 9. 

In another embodiment, the speech generating device is built into the mobile 
phone and may use the internal hardware, software and speaker system 1 1 , see 
figure 4. Existing telephones are usually provided with a microprocessor and a 

20 digital signal processor capable of being programmed to perform the required text 
to speech conversion. Thus, the text to speech conversion may be embodied as a 
software product, e.g. a computer program on a readable medium or deliverable 
through the Internet 

The microcontroller may for example be a commercially available circuit 

25 comprising a programmable flash memory, general purpose input/output lines and 
working registers, internal and external interrupts, a programmable serial universal 
asynchronous receiver and transmitter (UART) and a port for a serial peripheral 
interface. The registers are programmed to control the behaviour of the 
microcontroller in the desired way. The microcontroller is responsible for receiving 

30 the data to be converted to speech and sending the data to the TTS circuit 

The TTS circuit 7 may be a commercially available circuit The circuit 
should have an output designed to drive a speaker, and preferably also a telesocket 
for headphone or an external loudspeaker. To get a higher volume a general 
amplifier 8 could be used, e.g. a fully differential audio power amplifier. 

35 The TTS circuit should also support SMS (Short Message Service) and 

preferably a modifiable abbreviation list. The TTS circuit also should support 
various languages. In a preferred embodiment it is possible to program other 
languages through a serial port allowing the user to download different languages. 
A standard speaker voice is built-in, but preferably it is also possible to download 
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different speaker voices or connect external memories, for instance so called 
memory sticks, containing voice data. When the speech generating device is 
connected or integrated in a mobile phone or communicator, databases could be 
downloaded via the telecommunication network or the Internet 

5 The TTS circuit receives data to be read through its input port, e.g. ASCII 

characters, converts it into spoken audio and sends it to an analog output. A typical 
circuit comprises a text processor, a smoothing filter and multilevel memory storagie 
array. The voice and audio signals are stored in the memory in their natural, 
uncompressed form, which provides a good voice reproduction quality. 

10 The speech conversion is conventional and is not described in detail here. 

Briefly, the text-to-speech me chanis m comprises text normalisation, word to 
phoneme conversion and phoneme mapping. The text normalisation is the process 
of translating the incoming text to pronounceable words. It expands abbreviations 
and translates numeric strings to spoken words. The abbreviation list can be 

1 5 modified This enables flexibility of adding abbreviations specifically for the text, 
either by the developer or by the end user to customise the device. Even the unique 
characters of SMS are supported, me aning that icons such as smilies ;-) will be 
replaced by its corresponding true spoken meaning. This means that an SMS 
containing abbreviations and icons will be correctly recited 

20 The TTS circuit should have an internal input buffer that could hold at least 

256 characters in order to receive an entire SMS consisting of 160 characters. This 
means that no extra memory is needed in the connecting apparatus. 

The microcontroller 6 preferably is connected to a volume control to adjust 
the volume of a speaker system connected For instance, two buttons could be 

25 provided, one to increase the volume and one to decrease the volume. The buttons 
are suitably connected to the interrupt pins of the microcontroller. 

The speech generating device is provided with an interface for connecting 
the device to the phone via its system connector. The system connector interface 
comprises audio signals, two serial channels, power leads and the analog and digital 

30 ground leads. A typical system connector interface 10 is shown in fig. 2. 

The mobile telephone is arranged to extract texts and characters from the 
data shown on the display and to send it to the speech generating device. The 
extracted text string may be sent to the device to place the data on the system bus. 
All text strings are stored in a list and a text ID is a pointer used to point out the 

3 5 different text strings. 

Fig. 3 shows the data flow diagram between the blocks in the system. The 
different blocks need the right interfaces to communicate properly with each other. 
The interface between the phone 1 and the microcontroller 6 consists of a universal 
asynchronous receiver and transmitter UART, while the microcontroller 6 and the 
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TTS circuit 7 communicate via a serial peripheral interface. The UA^^m^fomi 
part of a commercial microcontroller. 

Fig. 4 shows an example of the operation of the present invention. The 
mobile phone 1 includes a display 2 currently showing part of a message, e.g. an 
5 SMS. The keypad includes scroll buttons 3 for moving in the display. Currently one 
line 4 of the display is marked by higbiighting the text. In an automatic mode, the 
control unit extracts one line or word after another at a fixed or adjustable rate and 
sends it automatically to the speech generating device for translating into spoken 
audio signals. It is preferably possible to pause, rewind and move fast forward in the 
10 text The speed of the speech reading the text can be adjusted to suit each 
individual. 

In another mode, the user scrolls in the display by means of the buttons 3 to 
select one line for sending to conversion circuit and reading aloud. The user may 
also select a whole text or a file, such as a message or downloaded article. The 

15 selected text is sent to the conversion circuit. 

In a further mode, the text to speech conversion is active when the user is 
writing a message, such as an SMS. After inputting a letter or sign, this is read 
aloud. When a whole word is finished, e.g. as triggered by the input of a space, the 
word is sent to the conversion circuit and read aloud. Further, when a punctuation 

20 mark is input the whole last sentence may be read, and finally the whole message 
may be read before it is sent The control unit sends the text to be read automatically 
in dependence of a definite set of characters, such as spaces and punctuation marks, 
and also, optionally, each input sign or letter. 

The text-to-speech conversion in the phone is not only an aid for the visually 

25 impaired and car drivers but also a step further in personalising the phone. Some of 
the possibilities with the text-to-speech function in a mobile telephone are: 

- Interaction with voice control. A voice command from the user can be 
used to control functions in the phone, like make a call or navigating in 
menus, and the speech function can then confirm the commands and 

30 possibly add help messages. 

- Extended help functions, giving spoken explanations to a selected topic, 
like a step-by-step instruction on how to install an e-mail account The 
whole instruction manual can be accessed in this way. This function can 
be activated and controlled by a shortcut or by voice recognition. 

35 - By saving texts on memory sticks connectable to the device or the 

mobile phone, it is possible to have huge text masses like books read 
Reading reminder and alerts from a calendar. 

- Reading pages and articles downloaded from the Internet or by WAP. 
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- Use as a navigation aid together with GPS (Global Positioning System) 

and the Yellow Pages route service. 
Different voices are possible. It is contemplated that popular voices like film 
stars etc. could be available for downloading or sold as connectable memory sticks. 
5 The spoken audio signal could also be combined with music files, e.g. MIDI 
(Musical Instrument Digital Interface) files. 

The invention may be implemented as a separate accessory connectable to an 
apparatus; or an apparatus incorporating such a device. The invention also relates to 
an apparatus connectable to such a device. The invention may be implemented by 
1 0 hardware or by software included in a self-contained apparatus or various 
combinations thereof. The scope of the invention is only limited by the claims 
below. 
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CLAIMS 

A device (5) for generating speech, characterised by: 
a microcontroller (6) connectable to an apparatus for receiving data to be 
converted to speech, and sending the data to a conversion circuit (7); 
a conversion circuit (7) connectable to a speaker system (9) for converting the 
data to a speech signal. 



A device according to claim 1, characterised in that the data is supplied as 
ASCII characters. 

A device according to claim 1 or 2, characterised in that the conversion 
circuit (7) supports various selectable languages. 



4. A device according to claim 3, characterised in that the conversion circuit 
(7) is capable of downloading languages via the connected apparatus. 

5. A device according to any one of claims 1 to 4, characterised in that the 
conversion circuit (7) supports various selectable voices. 

6. A device according to claim 5, characterised in that the conversion circuit 
(7) is capable of downloading voices via the connected apparatus (1). 

7. A device according to any one of claims 1 to 6, characterised in that the 
speed of the speech signal is adjustable. 

8. A device according to any one of claims 1 to 7, characterised in that the 
microcontroller (6) is connectable to a memory containing language 
information, such as various languages, abbreviation lists and dictionaries. 

9. A device according to any one of claims 1 to 8, characterised in that the 
microcontroller (6) is connectable to a memory containing voice settings. 

10. A device according to any one of claims 1 to 9, characterised in that the 
microcontroller (6) is connectable to the apparatus (1) by means of a system 
connector having an interface (10) for audio signals, serial channels, power leads 
and analog and digital ground leads. 

11. A device according to claims 1 0, characterised in that the device is 
implemented as a functional cover, comprising a shell covering the front of the 
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apparatus (1) and a microprocessor cooperating with the processor of the 
apparatus (1). 

12. A device according to any one of claims 1 to 1 1, characterised in that the 

5 connectable apparatus (1) is a portable telephone, a pager, a communicator or an 
electronic organiser. 

13. An apparatus (1) having a display (2) for showing various readable data, 
characterised by a control unit arranged to extract readable data for sending to a 

10 device (5) for generating speech in accordance with any one of the preceding 
claims. 

14. An apparatus according to claim 13, characterised in that the readable data 
includes texts from menus, text messages, help information, calendars or 

1 5 confirmation of actions taken with the apparatus (1 ). 

15. An apparatus according to claims 13 or 14, characterised in that the control 
unit is arranged to extract a part of the readable data, such as a line or a word, at 
a time from the display (2) and sending it automatically to the speech generating 

20 device (5) at a fixed or controllable rate. 

16. An apparatus according to claims 13, 14 or 15, characterised in that the 
control unit is arranged to extract a part of the readable data, such as a line or a 
word, at a time from the display (2) and sending it to the speech generating 

25 device (5) in dependence of scrolling in the display (2). 

17. An apparatus according to claims 13, 14, 15 or 16, characterised in that the 
control unit is arranged to extract a part of the readable data, such as a line or a 
word or a character, at a time from the display (2) and sending it to the speech 

30 generating device (5) in dependence of inputting characters to the apparatus. 

18. An apparatus according to claims 17, characterised in that the control unit is 
arranged to send readable data as triggered by the input of definite characters, 
such as letters, signs, spaces or punctuation marks. 

35 

19. An apparatus according to any one of claims 13 to 1 8, characterised in that 
the control unit is arranged to extract readable data from a selected file and 
sending it automatically to the speech generating device (5) at a fixed or 
controllable rate. 
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20. An apparatus (1) having a display for showing various readable data, 
characterised by including a control unit and a device for generating speech 
comprising a conversion circuit for converting data to a speech signal and 

5 connectable to a speaker system (9; 1 1), wherein the control unit is arranged to 
extract readable data for sending to the speech generating device . 

21. An apparatus according to claim 20, characterised in that the speaker 
system (1 1) is integrated with the apparatus. 

10 

22. An apparatus according to claim 20 or 2 1 , characterised in that the data is 
supplied as ASCH characters. 

23 . An apparatus according to claim 20, 2 1 or 22, characterised in that the 
15 conversion circuit supports various selectable languages. 

24. An apparatus according to claim 23, characterised in that the apparatus (1) 
is capable of downloading languages. 

20 25. An apparatus awarding to any one of claims 20 to 24, characterised in that 
the conversion circuit supports- various selectable voices. 

26. An apparatus according to claim 25, characterised in that the apparatus (1) 
is capable of downloading voices. 

25 

27. An apparatus according to any one of claims 206 to 26, characterised in that 
the speed of the speech signal is adjustable. 

28. An apparatus according to any one of claims 20 to 27, characterised in that 
30 the apparatus ( 1) is connectable to a memory containing language information, 

such as various languages, abbreviation lists and dictionaries. 

29. An apparatus according to any one of claims 20 to 28, characterised in that 
the apparatus (1) is connectable to a memory containing voice settings 

35 

30. An apparatus according to any one of claims 20 to 29, characterised in that 
the readable data includes texts from menus, text messages, help information, 
calendars or confirmation of actions taken with the apparatus (1). 
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31. An apparatus according to any one of claims 20 to 29, characterised in that 
the control unit is arranged to extract a part of the readable data, such as a line or 
a word, at a time from the display and sending it automatically to the speech 
generating device at a fixed or controllable rate. 

5 

32. An apparatus according to any one of claims 20 to 3 1 5 characterised in that 
the control unit is arranged to extract a part of the readable data, such as a line or 
a word, at a time from the display and sending it to the speech generating device 
in dependence of scrolling in the display (2). 

10 

33. An apparatus according to any one of claims 20 to 32, characterised in that 
the control unit is arranged to extract a part of the readable data, such as a 
character, a line or a word, at a time from the display (2) and sending it to the 
speech generating device (5) in dependence of inputting characters to the 

15 apparatus. 

34. An apparatus according to cl aims 33, characterised in that the control unit is 
arranged to send readable data as triggered by the input of definite characters, 
such as letters, signs, spaces or punctuation marks. 

20 

35. An apparatus according to any one of claims 20 to 34, characterised in that 
the control unit is arranged to extract readable data from a selected file and 
sending it automatically to the speech generating device (5) at a fixed or 
controllable rate. 

25 

36. An apparatus according to any one of claims 13 to 35, characterised in that 
the apparatus is a portable telephone, a pager, a communicator or an electronic 
organiser. 

30 37. A computer program product loadable into the internal memory of an 

apparatus (1) having a display for showing various readable data, characterised 
by comprising software code portions to achieve the functionality of the 
apparatus in accordance with any one of claims 20 to 36. 
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38. A computer program product according to claim 37 5 embodied on a computer 
readable medium. 
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